1. Field of the Invention
The present invention relates to a virtual object display device, method, non-transitory computer readable recording medium storing a program, and system capable of changing the display state of a virtual object when displaying the virtual object using augmented reality, for example.
2. Description of the Related Art
In recent years, a display system has been proposed that uses augmented reality that makes a virtual object appear as if it is present in real space by superimposing the virtual object on a real-time background image obtained by imaging the real space and displaying the result on a display device, such as a head mount display. In such a system, a marker that specifies a position where a virtual object is to be displayed is disposed in the real space. Then, the marker is detected from the background image obtained by imaging the real space. In addition, a display position, a display size, and a display direction of the virtual object are determined according to the position, size, and direction of the detected marker, and the virtual object is displayed on the display device according to the determined display position, size, and direction. As a marker, an image, such as a two-dimensional barcode, is used. In addition, a method using a light emission diode (LED) or a finger of an operator as a marker has also been proposed.
On the other hand, it is also possible to perform a certain operation on the displayed virtual object. For example, a method has been proposed in which a marker with various patterns drawn is imaged and an event associated with the pattern occurs when the marker displayed using augmented reality comes into contact with a virtual object (refer to JP2011-198150A). In the method disclosed in JP2011-198150A, an event occurs in which the displayed virtual object is deleted or replaced with another virtual object. In addition, a method in which a finger of an operator is included in a background image and the finger is moved like a cursor to operate a virtual object (refer to JP2013-105330A), a method in which an operation interface for operating a virtual object is displayed using augmented reality and a virtual object is operated using the displayed operation interface (refer to JP2013-172432A), and the like have been proposed.
In the medical field, attendees of surgery gather before the surgery, and a preoperative conference is held to explain the surgery. In such a preoperative conference, in recent years, the surgery is simulated by displaying a part as a surgery target using augmented reality. For example, in partial resection surgery of the liver, a virtual object of the liver is generated by extracting tissues, such as liver, a portal vein, a vein, an artery, a body surface, a bone, and a tumor, from tomographic images of a three-dimensional image obtained from a computed tomography (CT) image or a magnetic resonance imaging (Mill) image and visualizing these as a three-dimensional image. Then, the virtual object is displayed in an actual size using the augmented reality, and a lead surgeon who is the representative of the preoperative conference explains the surgery to the attendees of the conference and simulates the surgery using the displayed virtual object. At this time, each attendee of the conference wears a display device, such as a head mount display, so that everyone can hold the conference while watching the same virtual object.
In applications of the display system using such augmented reality to the medical field, a method has been proposed in which a virtual object of a surgery target and an actual object, such as a medical instrument, are superimposed on the head mount display and display and non-display of the virtual object are switched according to the instruction of the operator or the virtual object is enlarged or reduced according to a distance from the object (refer to JP2014-155207A). A method has also been proposed in which a virtual object is displayed on a head mount display to be worn by each person, an object such as a scalpel is detected, and enlargement, transmissive display, and the like of the virtual object are switched by operating the object (refer to WO2012/081194A). When displaying a virtual object with a marker as a reference, a method has been proposed in which a plurality of markers are displayed and a position, a direction and an inclination are changed for each corresponding object by moving the marker (refer to JP2014-010664A).